Skip to content

Use includes instead of system_includes for includes attr #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

arvi18
Copy link

@arvi18 arvi18 commented Apr 15, 2025

Previously even though the attribute was named includes it was passed
through the system_includes field of the compilation context. This
resulted in toolchains passing these include paths with -isystem,
which is unexpected if you use this for first party code.

Many non bazel-first projects have header directory structures that
require custom include paths be propagated throughout the graph, the
alternative to includes is to use strip_include_prefix. The downside
of strip_include_prefix is that you add 1 include path per
cc_library, even if the libraries are in the same package. With
includes these are deduplicated. In the case of LLVM using includes
reduced the number of search paths on the order of hundreds.

If users want to use -isystem for third party code that uses
includes, they can pass --features=external_include_paths --host_features=external_include_paths

If there are first party libraries users want to use -isystem with,
they can use features = ["system_include_paths"]

Fixes bazelbuild#20267

RELNOTES[INC]: Use -I instead of -isystem for cc_library / cc_binary includes attr. To use -isystem for only external repositories, you can pass --features=external_include_paths --host_features=external_include_paths. To use -isystem for a single cc_library / cc_binary includes, you can set features = ["system_include_paths"], on the target

Summary by CodeRabbit

  • New Features

    • Expanded support for specifying general include directories in C++ and Objective-C build rules.
    • Added new build flags to improve handling of external include paths during the build process.
  • Bug Fixes

    • Improved validation to correctly ignore external include directories during compilation.
  • Tests

    • Updated and added tests to verify correct handling and propagation of include directories and related compiler flags.
  • Chores

    • Updated internal scripts to pass new build options for enhanced compatibility and warning control on Windows.

Previously even though the attribute was named `includes` it was passed
through the `system_includes` field of the compilation context. This
resulted in toolchains passing these include paths with `-isystem`,
which is unexpected if you use this for first party code.

Many non bazel-first projects have header directory structures that
require custom include paths be propagated throughout the graph, the
alternative to `includes` is to use `strip_include_prefix`. The downside
of `strip_include_prefix` is that you add 1 include path per
`cc_library`, even if the libraries are in the same package. With
`includes` these are deduplicated. In the case of LLVM using `includes`
reduced the number of search paths on the order of hundreds.

If users want to use `-isystem` for third party code that uses
`includes`, they can pass `--features=external_include_paths --host_features=external_include_paths`

If there are first party libraries users want to use `-isystem` with,
they can use `features = ["system_include_paths"]`

Fixes bazelbuild#20267

RELNOTES[INC]: Use `-I` instead of `-isystem` for `cc_library` / `cc_binary` `includes` attr. To use `-isystem` for only external repositories, you can pass `--features=external_include_paths --host_features=external_include_paths`. To use `-isystem` for a single `cc_library` / `cc_binary` `includes`, you can set `features = ["system_include_paths"],` on the target
@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

this requires a toolchain change in order to add a test, I can do that if folks are happy with this direction

@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

see the alternative here bazelbuild#25751

@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

Selfishly, we have on the order of a dozen+ cc_toolchains that will need to add the "system_include_paths" feature for backwards compatibility until we can spend the time to test the rollout.

I'm not opposed in principle, but can we add an incompatible flag for this? Or keep the current behavior if a feature is unset?

Open to suggestions, but I'd like to have a way to disable this globally (for our monorepo) without having to modify many toolchains.

@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

totally fair, I can add this behind an incompatible flag, before I do that on this one, should we consider bazelbuild#25751 or should I close that and move forward with this approach?

@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

it looks like there is another implementation option, we can use feature_configuration.is_requested instead of is_enabled so that toolchains don't have to contain the feature, then I imagine it could be added in the REPO.bazel to enable globally, without any toolchain updates. What do you think about that option? The downside is still that you'd have to handle --features=external_include_paths --host_features=external_include_paths separately

@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

I've updated to the is_requested approach because the tests have the same issue

@arvi18
Copy link
Author

arvi18 commented Apr 15, 2025

ok the CI failures here lead to some interesting discoveries. for one the BUILD files of both grpc and c-ares have undeclared inclusion errors lurking that only hit if you use --spawn_strategy=standalone (which the bootstrap build of bazel does).

the change here is that previously those were ignored because --strict_system_includes=false (the default) disables those checks for any headers from -isystem, but they no longer are.

The fix here is to pass --features=external_include_paths --host_features=external_include_paths to continue using -isystem for external headers, and to start external_includes the same as system_includes for --strict_system_includes.

I've added a release notes blurb as well

@visz11
Copy link
Collaborator

visz11 commented Apr 25, 2025

@coderabbitai full review

Copy link

coderabbitai bot commented Apr 25, 2025

✅ Actions performed

Full review triggered.

Copy link

coderabbitai bot commented Apr 25, 2025

Walkthrough

This set of changes updates the handling of include directories in Bazel's C++ and Objective-C build rules. The code and build scripts are modified to shift from using "system include" directories to more general "include" directories in both the build logic and associated tests. Several scripts and Starlark files now use includes instead of system_includes, and helper functions are renamed accordingly. Compiler and Bazel flags are updated in shell scripts to accommodate new include path features. Tests are adjusted to reflect the new include handling, with new assertions and test cases added.

Changes

Files Change Summary
compile.sh, scripts/bootstrap/bootstrap.sh Added/updated Bazel build flags for external include paths and compiler options to control warning levels and include path features.
src/main/java/com/google/devtools/build/lib/rules/cpp/CppCompileAction.java Expanded the set of directories ignored during include validation to include external include directories.
src/main/starlark/builtins_bzl/common/cc/cc_binary.bzl, src/main/starlark/builtins_bzl/common/cc/cc_library.bzl, src/main/starlark/builtins_bzl/common/objc/compilation_support.bzl Changed calls from using system_include_dirs to include_dirs for passing include directories to compilation functions.
src/main/starlark/builtins_bzl/common/cc/cc_helper.bzl Renamed _system_include_dirs to _include_dirs, updated struct keys, and added a deprecation note.
src/main/starlark/builtins_bzl/common/cc/compile/cc_compilation_helper.bzl Modified logic to merge or separate system and general include directories based on feature configuration.
src/test/java/com/google/devtools/build/lib/rules/cpp/CcCommonTest.java Updated tests to check general include directories instead of system include directories; added a new test for include handling.
src/test/java/com/google/devtools/build/lib/rules/cpp/CcLibraryConfiguredTargetTest.java Updated assertions to check for -I flags and general include directories; added assertion to ensure implementation deps' includes are not propagated.
src/test/java/com/google/devtools/build/lib/rules/cpp/StarlarkCcCommonTest.java Adjusted assertions to reflect the new logic for merged include directories.
src/test/java/com/google/devtools/build/lib/rules/objc/ObjcLibraryTest.java Changed expected compile argument from -isystem to -I for transitive C++ dependency includes.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Bazel
    participant StarlarkRule
    participant CppCompileAction

    User->>Bazel: Starts build with updated flags
    Bazel->>StarlarkRule: Invokes cc_binary/cc_library/objc rules
    StarlarkRule->>StarlarkRule: Calls cc_helper.include_dirs(...)
    StarlarkRule->>CppCompileAction: Passes include dirs as 'includes'
    CppCompileAction->>CppCompileAction: Ignores built-in, system, and external include dirs during validation
    CppCompileAction-->>Bazel: Compilation proceeds with updated include handling
Loading

Assessment against linked issues

Objective Addressed Explanation
Rename includes to system_includes in cc_* functions (bazelbuild#20267) The code changes do the opposite: they replace system_includes with includes in function arguments and logic, rather than renaming includes to system_includes.

Poem

Oh, what a tangle of paths we weave,
When headers and includes we must retrieve!
From system to general, the flags realign,
Directories merge, and the build scripts shine.
With every hop and every bound,
This bunny keeps code safe and sound.
🐇✨

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.31.1)
src/test/java/com/google/devtools/build/lib/rules/cpp/StarlarkCcCommonTest.java

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c0c98a2 and c9a5bfb.

📒 Files selected for processing (12)
  • compile.sh (1 hunks)
  • scripts/bootstrap/bootstrap.sh (1 hunks)
  • src/main/java/com/google/devtools/build/lib/rules/cpp/CppCompileAction.java (1 hunks)
  • src/main/starlark/builtins_bzl/common/cc/cc_binary.bzl (1 hunks)
  • src/main/starlark/builtins_bzl/common/cc/cc_helper.bzl (2 hunks)
  • src/main/starlark/builtins_bzl/common/cc/cc_library.bzl (1 hunks)
  • src/main/starlark/builtins_bzl/common/cc/compile/cc_compilation_helper.bzl (1 hunks)
  • src/main/starlark/builtins_bzl/common/objc/compilation_support.bzl (1 hunks)
  • src/test/java/com/google/devtools/build/lib/rules/cpp/CcCommonTest.java (4 hunks)
  • src/test/java/com/google/devtools/build/lib/rules/cpp/CcLibraryConfiguredTargetTest.java (2 hunks)
  • src/test/java/com/google/devtools/build/lib/rules/cpp/StarlarkCcCommonTest.java (1 hunks)
  • src/test/java/com/google/devtools/build/lib/rules/objc/ObjcLibraryTest.java (1 hunks)
🔇 Additional comments (16)
compile.sh (1)

63-64: Added MSVC external warning suppression flags for Windows builds

This change adds the flags --copt=/external:W0 and --host_copt=/external:W0 to suppress warnings from code in external include paths when building on Windows. This complements the external_include_paths feature flags added in bootstrap.sh.

scripts/bootstrap/bootstrap.sh (1)

46-47: Added feature flags for external include paths

The flags --features=external_include_paths and --host_features=external_include_paths enable the new behavior for handling external include paths during both target and host builds. This ensures that external dependencies continue using -isystem flags while first-party code uses -I flags, aligning with the PR objectives.

src/main/starlark/builtins_bzl/common/cc/cc_helper.bzl (1)

1065-1068: Function renamed from _system_include_dirs to _include_dirs

The function has been renamed to better reflect its purpose, with both the new and old names pointing to the same implementation. This is part of the broader change to make Bazel use -I instead of -isystem for the includes attribute, aligning with developer expectations for first-party code.

The TODO comment about removing the old name is appropriate for maintaining backward compatibility while allowing for future cleanup.

src/test/java/com/google/devtools/build/lib/rules/cpp/StarlarkCcCommonTest.java (2)

1401-1401: Modified assertion to reflect new include handling

The assertion on mergedSystemIncludes has been simplified to only check for "foo/bar", removing the check for "a/dep1/baz" and "a/dep2/qux". This aligns with the PR objective of changing how the includes attribute is handled - now the includes attribute uses -I instead of -isystem.


1405-1405: Correctly moved paths from system includes to regular includes

The assertion on mergedIncludes now checks for the presence of "a/dep1/baz" and "a/dep2/qux" in addition to "baz/qux". This correctly validates that these paths are now treated as regular includes rather than system includes, which matches the intended behavior changes in this PR.

src/main/java/com/google/devtools/build/lib/rules/cpp/CppCompileAction.java (1)

1135-1139: Expands ignored validation directories to include external includes.

This change adds external include directories to the list of directories ignored during include validation, alongside system include directories. This aligns with the PR's objective of using -I instead of -isystem for first-party code while still properly handling validation checks.

src/test/java/com/google/devtools/build/lib/rules/objc/ObjcLibraryTest.java (1)

1400-1400: Updates test to verify includes are passed with -I instead of -isystem.

This test change verifies that include directories from transitive C++ dependencies are passed to the compile action with -I flags instead of -isystem flags, which is consistent with the PR's goal of changing how include directories are handled.

src/main/starlark/builtins_bzl/common/cc/cc_library.bzl (1)

66-66: LGTM: Changed from system_includes to includes in cc_common.compile

This change correctly modifies the cc_library rule to use regular include directories (-I flags) instead of system include directories (-isystem flags) for the paths specified in the includes attribute. This change aligns with user expectations for first-party code and helps reduce include path duplication.

src/main/starlark/builtins_bzl/common/objc/compilation_support.bzl (1)

108-108: LGTM: Updated to use include_dirs instead of system_include_dirs

This change consistently applies the same pattern used in C++ rules to Objective-C compilation, ensuring that include paths are passed with -I instead of -isystem, which is appropriate for first-party code.

src/main/starlark/builtins_bzl/common/cc/cc_binary.bzl (1)

513-513: LGTM: Switched from system_includes to includes in cc_common.compile

This change correctly applies the include path handling update to cc_binary rules, ensuring consistency across all C++ rules. The include directories will now be passed using -I flags instead of -isystem flags.

src/test/java/com/google/devtools/build/lib/rules/cpp/CcLibraryConfiguredTargetTest.java (1)

1908-1913: Updated assertion to verify implementation of include dependency handling.

The change correctly adds a new assertion to verify that implementation_dep's include directory is not propagated as a regular include directory. This aligns with the broader change of using -I instead of -isystem for includes attribute.

src/main/starlark/builtins_bzl/common/cc/compile/cc_compilation_helper.bzl (1)

230-233: Implements conditional logic for include path treatment.

This change implements the core functionality to control how include directories are processed. When the "system_include_paths" feature is requested, include directories are treated as system includes (with -isystem compiler flag), maintaining backward compatibility. Otherwise, the default behavior now uses regular include directories (with -I compiler flag).

This aligns with the PR objective of changing includes attribute to use -I instead of -isystem by default, while providing a way to opt back into the old behavior.

src/test/java/com/google/devtools/build/lib/rules/cpp/CcCommonTest.java (4)

576-579: Updated assertion to check include directories instead of system include directories.

The test is updated to verify include paths are now added to regular include directories rather than system include directories, aligning with the new default behavior.


599-605: Updated genfiles directory inclusion test.

This correctly updates the test to check inclusion in regular include directories rather than system include directories, consistent with the new behavior of the includes attribute.


629-630: Added "system_include_paths" feature to test the legacy behavior.

The test now adds the "system_include_paths" feature to verify that when this feature is enabled, include directories are still treated as system includes. This confirms the feature flag works as expected.


648-685: Added new test for default include behavior.

This new test case properly verifies the default behavior (without the "system_include_paths" feature), confirming that include directories specified via the includes attribute are added as regular include paths with -I flags.

The test complements the existing test for legacy behavior, providing full coverage of both the new default and the opt-in legacy behavior.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Request: rename includes to system_includes in cc_* functions
3 participants